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Dear Colleagues: 

What follows are the Proceedings for the 33 ld Annual Conference of the North East Association 
for Institutional Research (NEAIR) which was held in Philadelphia, Pennsylvania. The 
Proceedings include a variety of papers and presentations from that conference. We hope they 
will serve as a reference and resource for future professional endeavors. Congratulations go to 
Sally Lesik, winner of this year’s Best Paper Award, and Michelle Appel and Chad Muntz, 
winners of this year’s Best IR Report/Practitioner Paper Award. 

“Thanks” go to all who worked so hard to make the Philadelphia Conference the success that it 
was. The program as well as the central site attracted over 300 attendees, a record for NEAIR. 

Particular thanks go to Nancy Ludwig who served as the conference’s Program Chair. Nancy 
spent countless hours on all aspects of the program to insure that it was the best that it could be - 
securing stimulating and thought provoking keynote speakers; to reviewing the proposals; to 
making sure there was broad participation and something of interest for each of the different 
segments of Higher Education. Thanks also go out to Ellen Peters (Associate Program Chair) 
who assisted Nancy in carrying out these duties. 

Also, special tha nk s go out to Allison Walters who served as Local Arrangements Chair for this 
conference. She was able to solicit and secure computer and technical support from Drexel 
University’s Le Bow College of Business resulting in significant savings to the association and 
enabling computer-aided workshops to be held on-site. In addition, she worked tirelessly with 
the hotel and volunteers to make sure our presenters and attendees had an enjoyable experience 
in all aspects of their stay. 

A heartfelt “Thank You” goes to Beth Simpson, who not only provides administrative services 
to NEAIR but also provides substantial support to the President, Program Chair, and Local 
Arrangements Chair by making sure all bases are covered and attended to numerous details. 

Beth also serves as the cheerful and helpful welcoming committee when you arrive at the 
conference registration desk. 

A large “Tha nk You” goes to Cindy Clarke, Pre-conference Workshops Coordinator, for the 
great job she did in Philadelphia along with lots of good luck on her position as Program Chair 
for the upcoming NEAIR conference in New Brunswick. 

I would like to acknowledge the fine work of Roland Hall and Annemarie McMuIlin for 
making sure the association and conference websites were getting our message out in a timely 
and accurate way. 
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“Many Thanks” also go to Tim Walsh, Vendor Coordinator, for coordinating the participation of 
a fine group of vendors at the conference. 

“Thank You” Cathy Alvord, Publications Chair, for your excellent work throughout the year as 
well as your work on these Proceedings. 

Also appreciated are the valuable contributions of those who volunteered their time to review 
papers (Marilyn Blaustein) and poster submissions (Kathy Keenan) as well as staff the 
registration desk and lead a dinner groups. “Thank you” to the Evaluation Coordinators (Mindy 
Wang and Shu-Ling Chen). 

And last, but not least, “Thank You” to all members of the 2005/2006 Steering Committee for 
your help and support in the conference planning process and other association business 
throughout the year. 

I look forward to seeing you all in New Brunswick in 2007! 



Marge Wiseman 
President, NEAIR 2005 - 2006 
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The Pluses and Minuses of Policy Analysis: 1 
Predicting the Impact of a New Grading Policy 2 



Michelle Appel 
Chad Muntz 

Office of Institutional Research and Planning 
University of Maryland 



Executive Summary 

The purpose of this study was to better understand the impact of the new weighted 
plus/minus grading calculation on undergraduate graduation rates and course availability. 

Grades from the fall 2002 cohort of incoming undergraduate students were used to determine the 
impact on probation and dismissal with the accompanying impact on graduation rates. 
Additionally, all undergraduate courses assigning the C- grade during the 2003-04 and 2004-05 
Academic Years were examined to understand course availability. 

Findings 

Impact on Graduation Rates and Cumulative GPAs: 

♦♦♦ Although semester GPA changes appear to have a relatively balanced impact (similar 
numbers of students positively and negatively impacted) over the long term , most 
students’ cumulative GPAs are affected negatively. After seven semesters, only the 
band of students with a GPA of 3.9 or above had a majority of the students who would be 
positively impacted by the policy change. The vast majority of students at all other GPA 
ranges would be hurt by the policy change. 

♦♦♦ Disciplinary actions would increase under the new policy. These findings are consistent 
with another analysis conducted by the Office of the Registrar, November 2003. The 
impact on disciplinary actions, particularly dismissals, also appears to be amplified 
when examined over time rather than for a single semester. Raising a cumulative GPA 
appears to be more difficult under the new plus/minus weighting system. 

♦♦♦ The number of dismissals more than doubled each semester after the first year, impacting 
the number of students who could persist to graduation. This could lower the graduation 
rate by approximately 2 percentage points if all of these dismissed students would have 
continued through graduation. 

❖ The potential impact for dismissal under the new grading system is more pronounced for 
minority students (particularly because the base is small). African American dismissals 
could decrease the graduation rate by as many as 7 percentage points. Hispanic rates 
could decrease by 5 percentage points. 



1 NEAIR 2006 Best IR Practitioner Award. 

2 This paper was presented to demonstrate a methodological approach to be used when predicting the impact of a 
new policy. Some results contained within this paper have been modified to mask individuals or units who may be 
impacted; this is intended to allow the presenters to demonstrate the process while protecting the confidentiality of 
units within the University. 
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♦♦♦ Eliminating the C-, D+, and D- grades could reduce the impact on African American 
graduation rates to a 5 -point decline. The decline in the overall graduation rate would 
not, however be impacted, as these grades make up a relatively small portion of the grade 
distribution. 

Impact on Course Demand and Its Financial Implications: 

♦♦♦ Course Demand may change dependent on the acceptance of minus grades for fulfilling 
CORE and major requirements. If a minus grade is not deemed acceptable, this will 
increase course repeats, putting additional stress on course availability. 

♦♦♦ As many as 40 additional CORE sections per semester may be required to 

accommodate the increased demand. Additional course section units (as many as 40 
more) may be required to accommodate major requirements or other prerequisites. 
Current waitlists may make this more pronounced. 

♦♦♦ If this were additional demand (versus a shift from demand for other courses) this could 
cost the University upwards of $320,000 per fiscal year at $4,000 per section. 

♦♦♦ Many CORE courses are held in large lecture halls. Space availability and utilization 
constraints may impact the University’s ability to offer additional course sections. 

Limitations 



This study likely represents a worst-case scenario. 

• All analyses assume that student behavior will not change under the new grading weights 
and that faculty will continue to utilize the plus minus system as they have in the past. 

• The calculations on graduation rate impact assume that the additional dismissals would 
persist to graduation; this may overstate the impact on the graduation rates. 

• The replacement of repeat course grades within the first 24 credits has not been 
incorporated into the semester GPA calculation. 

• The calculations on course demand and availability examine the C-, assuming that a C 
(2.0) is the minimum passing grade. Although this is not a CORE requirement per se, it 
is often a requirement for pre-requisite courses and courses that simultaneously fulfill 
both major and CORE requirements. 

• The course demand model assumes that all students achieving the C- would attempt to 
take the same course again; major changes, student motivation, and course repeat limits 
may combine to lower the actual demand. 

• Course demand calculations combined partial demand for sections into a total that may 
overstate the number of sections that would actually be required. They also did not 
account for potential shifts in demand that may occur due to the potential increased repeat 
behavior. 

• Several additional areas of impact exist that were not considered in this analysis, 
including the impact on graduate students, implications for Financial Aid recipients, and 
the impact on students applying to graduate programs. 

Purpose 

On December 12, 2005, the University Senate passed a policy assigning graduated 
numerical values to plus and minus grades. Prior to this policy, the University of Maryland 
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assigned the same weight to all grades with the same letter, regardless of the plus or minus 
values assigned by the faculty. The purpose of the current analysis was to identify any potential 
financial and academic impacts for the University of Maryland and its undergraduate students, 
given the new plus/minus grade weights. 

Introduction 

Reports from Eastern Kentucky University (2003), North Carolina State University 
(1997), Georgia State University (1998), Western Illinois University (2005), and Wake Forest 
University (1997) were gathered to examine the research on plus/minus grading. A short 
summary of the general findings is presented here as a context in which to understand this impact 
analysis. 

The positives of plus/minus grading systems identified in these studies were: the 
incentive to work hard all tenn; the rewards for students at multiple levels; the ability to better 
distinguish the quality of graduates; an increase in grading accuracy; and a lack of overall GPA 
change at the institution for any given semester. The use of plus/minus has been promoted as a 
potential aid for graduate school admissions and a tool to help committees distinguish between 
students. 

The negatives identified were: the decrease in the number of 4.0 graduates, an increased 
negative impact on lower GPAs, a negative impact on 3.5 - 4.0 students (typically scholarship 
students), increased grade appeals, and increased grade changes. One study included results of a 
survey, which indicated that students perceived the potential for loss of scholarship or other 
merit-based aid as a negative impact. 

One area still debated is the effect on grade inflation. Overall, many studies reported no 
overall institutional GPA change. However, it was noted that these studies did not address the 
inherent grading processes - which may not always be transparently objective - weakening their 
ability to conclusively address grade inflation in general. 

An area unaddressed by these studies was the cumulative impact on the student body. 
Many studies utilized a point-in-time analysis of semester grades. Additionally, financial 
implications beyond administrative costs to the institution were not addressed. 

Method 

There were two types of analyses conducted to investigate the impact of plus/minus 
grades - the first focused on the new student cohort of fall 2002, while the second explored the 
potential impact of C- grades on course demand. The fall 2002 cohort was used to investigate: 1) 
the change in student cumulative GPAs, and 2) the impact of C- on disciplinary actions of 
probation and dismissal for new freshmen. 
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Graduation Rate and Cumulative GPA: 



For all students that entered in fall 2002, both new freshman and new transfers, semester 
GPAs and Cumulative GPAs were re-calculated based on the new weights. Grades from all 
applicable UM courses (including those taken in the summer or taken prior to fall 2002) were 
included in the calculation of the GPA. Courses were deemed applicable if they met the 
following criterion: regular grading method (e.g., not pass/fail or audit), officially enrolled 
course, completed course (i.e., not withdrawn or incomplete), and designation as applicable 
within the data warehouse (including applicable toward degree requirements); credit by exam 
and courses that are not official UM courses were eliminated from the analysis. 

Cumulative and semester quality points were computed using the new weights passed by 
the Senate (A+=4.3, A=4, A-=3.7, B+=3.3, etc.). These quality points were then divided by 
cumulative and semester attempted credits (based on the credits from included courses) to 
determine the re-calculated GPA. Additional analyses eliminated the C-, D+, and D- grades by 
assigning the whole number quality points associated with the letter grade. 

Academic actions for each semester were then re-computed. Students with a cumulative 
GPA of 2.0 or better were deemed in good standing. Those students with a cumulative GPA 
below 2.0 were then assigned the academic action of either probation or dismissal, depending on 
their prior and semester academic performance: a) students who were previously in good 
standing were assigned probation for their first semester with a cumulative GPA below 2.0; b) 
students with over 60 credits who were previously on probation were assigned to the dismissal 
category; c) students with under 60 credits who were previously on probation were assigned to 
dismissal if their semester GPA was below 2.0 or to continuing probation if their semester GPA 
was 2.0 or better. 

Course Demand: 



To model course demand, the unit of analysis was the course section. All courses in 
which any student earned a grade of C- during the 2003-04 and 2004-05 academic years were 
examined to determine the potential impact of C- grades on course demand. Analyses examined 
the number of C- grades, the length of the course waitlist (if any), and the type of course (e.g., 
CORE) to understand the potential impact of course repeats due to C- grades. 

To determine the demand generated by students repeating courses in which they earned a 
C-, potential additional section units were calculated. The number of C- grades earned was 
divided by the average section size for the course to determine a section unit for that course. 
Additional analyses included the number of students on the waitlist by adding them to the 
numerator. These were then summed to calculate the total section units required: 

Total Section Units c- Only = £ (# of C- grades / average course section size). 

Total Section Units c-+ waitlist = 2 ([# of C- grades + waitlist]/ average course section size) 
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Assumptions/Methodological Caveats: 

This analysis assumes that student behavior will not change under the new grading 
system. Inherent in the calculations on graduation rate impact is the assumption that the 
additional dismissals would persist to graduation; this may overstate the impact on the 
graduation rates. Additionally, the replacement of repeat course grades within the first 24 credits 
has not been incorporated into the semester GPA calculation (though it has, to the best of our 
knowledge, been accounted for in the cumulative GPA). The dismissals for spring 2003 were 
calculated off of a 2.0; the actual dismissals were calculated with a slightly lower GPA during 
implementation of the new Probation and Dismissal policy. 

The calculations on course demand and availability may simplify the complex nature of 
student course selection and availability. They examine the C-, assuming that a C (2.0) is the 
minimum passing grade. Although this is not a CORE requirement per se, it is often a 
requirement for pre-requisite courses and courses that simultaneously fulfill both major and 
CORE requirements. For example, the Criminal Justice major requires MATH 1 1 1 or higher to 
be completed with a C or better; CJIS majors must also complete supporting sequence courses, 
frequently including CORE courses such as AMST 201, with a C or better. 

These analyses also combined partial demand for sections (e.g., the need for .25 section 
units) into an overall total. To the extent that small course units (such as .1 section units) would 
not necessitate an additional section, this may overstate the number of sections that would 
actually be added. However combining multiple large section units (such as a series of three 
courses, each requiring .8 section units that may add to 2.4) may understate the need. The model 
assumes that, on balance, summing course units provides a reasonably accurate estimate of 
demand. Additionally, currently existing waitlists may be exacerbated by the increased demand. 

Results 

Impact on Graduation Rates and Cumulative GPA: 

1) Changes in Semester GPAs under the new policy (versus the actual semester GPAs under 
the old policy) appear to evenly impact students, with similar numbers of negative change 
and positive change. In any given semester, there are many students both helped and 
hurt by the policy change when examining their semester GPA. See Appendix, Analysis 
la. 

2) Over the long term, almost all cumulative GPAs are affected negatively. The lower the 
cumulative GPA, the higher percentage of students affected negatively. After seven 
semesters, only the students with GPAs of 3.9 or above had a majority of students who 
had been helped by the policy change. The vast majority of students at all other GPA 
ranges were hurt by the policy change. See Appendix, Analysis la. Eliminating the C-, 
D+, and D- grades does not appreciably change this phenomenon; see Appendix, 

Analysis lb. 
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3) Disciplinary actions would likely increase under the new policy. These findings are 
consistent with another analysis conducted by the Office of the Registrar, November 
2003. See Appendix, Analysis 2. 

Because probation is more sensitive to single semester changes, while dismissal is more 
sensitive to cumulative academic performance, dismissals are more likely to be adversely 
impacted by the new policy. Over the long term, raising a cumulative GPA seems to be 
more difficult under the plus/minus system. The number of dismissals more than doubled 
each semester. This could lower the graduation rate by 2 percentage points, assuming all 
of these students would have continued to graduation. 

The impact of the new policy on dismissals is likely to differentially impact minorities. 
The increased number of African-American dismissals could lower that graduation rate 
by 7 percentage points; additional Hispanic dismissals could lower the graduation rate by 
5 percentage points. 

Eliminating the C-, D+, and D- grades could mitigate the impact on African American 
and graduation rates but would not likely reduce the impact on the overall graduation rate 
or on the Hispanic graduation rate. Under this alternative, the African American 
graduation rate could decline by 5 percentage points rather than 7. Although it would 
intuitively seem that this would have a more significant impact, the small number of C- 
and D-grades, relative to the universe of all grades, makes it difficult for this alternative 
to have a large impact. This alternative also negates any positive impact that these 
students may derive from the D+ grades, although the number of these is minimal. 

4) Over 1,000 students who began in fall 2002, predominantly those who began as New 
Transfers, have already graduated; 1% of these students would not have had the required 
2.0 for graduation under the new policy. Should C-, D+, and D- grades be eliminated, 
only 0.5% of graduates would be impacted. 

Impact on Course Demand: 

1) Course Demand may change dependent on the acceptance of minus grades. Under the 
current policy, where all letter grades are given the same weight, a “minus” grade is 
considered acceptable for completion of CORE and major requirements. If the minus 
grade is no longer accepted for these requirements (i.e., the requirement is the whole 
number numeric equivalent) course repeats may increase. 

2) Each semester, more than 2,500 grades of C- are given in about 700 courses. The new 
policy could require repetition of this course work depending upon major requirements. 
Overall, this yields a cumulative course repeat impact of about 100 Section Units each 
semester. The demand may be absorbed by already existing capacity in those courses 
without waitlists. 

3) If the same analysis considers only courses that typically have waitlists, there is still 
appreciable demand from students required to repeat the course. The total section unit 
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demand generated by C- students for courses with a waitlist is about 80 section Units 
each semester. This does not include already existing demand reflected in the waitlist. 

4) Students earning a C- yielded demand of about 40 additional section units needed each 
semester in CORE courses. Current seat availability and course waitlists may impact 
this demand. Because many CORE courses are held in large lecture halls, space 
availability and utilization constraints may also impact the University’s ability to offer 
additional course sections. 

5) The potential course demand created by the minus grades could have significant financial 
implications for the University, if additional sections were required. At a minimum, 
current resource allocation may be impacted because the need to repeat courses may alter 
already existing course taking patterns, shifting or changing the demand patterns that 
currently exist. If this were additional demand (versus a shift from demand for other 
courses ) this could cost the University upwards of $320,000 per fiscal year (calculated 
at $4,000 per section for 40 sections per semester). 

Limitations/Criticism 

• This analysis considered only the impact on the undergraduates and undergraduate 
courses. Similar analysis of the impact on graduate students and courses would be 
required to fully understand the impact on all students at the University. 

• This analysis was limited in its focus. Additional areas of impact to students include 
Financial Aid (e.g., would recipients of merit based aid remain eligible) and 
consideration for graduate school (e.g., would the student’s merit in the application 
process be impacted) as well as many others. 

Graduation Rate Model Limitations: 



• The model assumes that student behavior will not change. One purpose of plus/minus 
grades was to motivate all students in the grade distribution to work hard all semester 
long. It might be posited that C- students will rise to the expectation of C grades. 

• Inherent in the calculations for changes in graduation rates is the assumption that the 
additional dismissals would persist to graduation; this may overstate the impact on the 
graduation rates. 

• The replacement of repeat course grades within the first 24 credits has not been 
incorporated into the semester GPA calculation (though it has, to the best of our 
knowledge, been accounted for in the cumulative GPA). 

• The dismissals for spring 2003 were calculated off of a 2.0; the actual dismissals were 
calculated with a slightly lower GPA during implementation of the new Probation and 
Dismissal policy. 

Course Demand Model Limitations: 



• The sum of Course Units from C- students are made up of small fractions that could be 
absorbed next term in the non-wait-list and wait-list courses. 
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• C- Students may shift demand by requiring more sections offered in core or lower-level 
courses. If so, demand for upper level and sequential courses may be lower, which will 
allow those instructors to switch their teaching load. 

• To the extent that the requirement is a D ( 1 .0) rather than a C, this methodology may 
overstate demand. 



Appendix 

Analyses 1.A Tables: Effect of new policy on Cumulative GPAs for New Fall 2002 
Students. 

* Note: Percent bars may not total 100% because students without GPA changes are not 
represented with a bar. 
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Impact of Fall 2004 Cumulative GPA Change Under 
New Policy 



- 2 
° ID 
ui 

> c 
.•s ra 
in .c 
o O 
Q- <u 
ra .2 
— . <*-• 

> 0 ■ 
« o> 

s? a) : 



100 . 0 % 

80.0% 

60.0% 

40.0% 

20 . 0 % 

0 . 0 % 




^ n 1 ) ^ ^ O' _b> <b 



vO <v <V V O'- 'V ‘b- <b- 'b- < b\ ( X f 
‘V' v 5 V 3 0/ T? r b N v 3 'V 3 <b^ 

of 

% <V 

Original GPA Ranges 



* 



□ Pos% 
■ Neg% 



Impact of Fall 2004 Cumulative GPA Change Under 
New Policy 



0) 

> 


o 

0 


< 


— 


U) Q_ 


(/) 

o 


C 

0 


0 


Q. 


.C 




03 


o 


L. 

0 




0 

> 


i- 

1- 

D 


s— 

0 


0 


o 


n 


U) 


i— 


£ 


0 


0 


D 


z 


.c 


Z 


J— 

o 








12 




Impact of Spring 2005 Cumulative GPA Change 
Under New Policy 
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Analyses l.B Tables: Effect of ALTERNATE new policy on Cumulative GPAs for New 
Fall 2002 Students (2.0 for C- and all Ds have a 1.0) 



* Note: Percent bars may not total 100% because students without GPA changes are not 
represented with a bar. 
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Impact of Fall 2003 Cumulative GPA Change Under 
Alternate New Policy 
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Impact of Spring 2004 Cumulative GPA Change 
Under Alternate New Policy 
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Impact of Spring 2005 Cumulative GPA Change 
Under Alternate New Policy 
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Impact of Fall 2005 Cumulative GPA Change Under 
Alternate New Policy 
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Analyses 2 Table: Academic Dismissals 



TOTAL DISMISSED BY RACE 





Total 


American 

Indian 


Asian 


African 

American 


Hispanic 


White 


All 

Other 


Total New Freshmen 
Current Number 


4000 


25 


475 


450 


300 


2500 


250 


Dismissed 

% of New Freshmen 


200 


5 


25 


75 


25 


75 


25 


Dismissed 


5% 


20% 


5% 


17% 


8% 


3% 


10% 


New Policy 


300 


10 


50 


100 


50 


125 


50 


Change of New Policy 
% Dismissed Under 


100 


5 


25 


25 


25 


50 


25 


New Policy 


8% 


40% 


11% 


22% 


17% 


5% 


20% 


Alt New Policy 
Change of Alternate 


250 


10 


40 


90 


40 


100 


40 


New Policy 
% Dismissed Alt New 


50 


5 


15 


15 


15 


25 


15 


Policy 


6% 


40% 


8% 


20% 


13% 


4% 


16% 



NOTE: The numbers above are not actual - they are intended to illustrate the process and not actual UM results 
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Analysis 3 Tables: Section Unit Increases 



Use of C- Grades and Approximate Impact (in Section Units) from Required Repeats 



Total Section Units by Colleges 





Fall 03 


Spring 04 


Fall 04 


Spring 05 






# 


AVG 


Sec 




# 


AVG 


Sec 




# 


AVG 


Sec 




# 


AVG 


Sec 


College 


C- 


Crs 


Sec 


Unit 


C- 


Crs 


Sec 


Unit 


C- 


Crs 


Sec 


Unit 


C- 


Crs 


Sec 


Unit 


A 


75 


25 


30.0 


2.5 


100 


25 


40.0 


2.5 


75 


35 


35.0 


2.1 


50 


25 


30.0 


2.7 


B 


40 


10 


45.0 


0.9 


25 


5 


40.0 


0.6 


40 


10 


50.0 


0.8 


25 


10 


45.0 


0.6 


C 


800 


250 


30.0 


26.7 


800 


250 


30.0 


26.7 


850 


250 


30.0 


28.3 


750 


250 


25.0 


30.0 


D 


275 


50 


35.0 


7.9 


225 


35 


35.0 


6.4 


200 


40 


35.0 


5.7 


175 


40 


35.0 


5.0 


E 


600 


125 


50.0 


12.0 


600 


125 


50.0 


12.0 


625 


125 


50.0 


12.5 


575 


125 


55.0 


10.5 


F 


300 


50 


40.0 


7.5 


235 


50 


40.0 


5.9 


300 


40 


35.0 


8.6 


200 


40 


40.0 


5.0 


G 


475 


75 


40.0 


11.9 


400 


100 


35.0 


11.4 


425 


100 


35.0 


12.1 


425 


100 


35.0 


12.1 


H 


75 


25 


40.0 


1.9 


50 


25 


35.0 


1.4 


50 


25 


35.0 


1.4 


50 


25 


35.0 


1.4 


I 


250 


75 


40.0 


6.3 


225 


50 


30.0 


7.5 


275 


75 


35.0 


7.9 


225 


60 


35.0 


6.4 


J 


100 


25 


50.0 


2.0 


100 


25 


50.0 


2.0 


100 


25 


45.0 


2.2 


100 


30 


45.0 


2.2 


K 


25 


10 


30.0 


0.8 


25 


10 


40.0 


0.6 


20 


10 


35.0 


0.6 


25 


10 


40.0 


0.6 


L 


50 


25 


25.0 


2.0 


10 


10 


35 


0.29 


20 


10 


25.0 


0.8 


20 


10 


30.0 


0.67 


TOTAL 

(ALL) 


3065 


745 


37.9 


82.2 


2795 


710 


38.3 


77.4 


2980 


745 


37.1 


83.1 


2620 


725 


37.5 


76.2 



C- is the number of students that received a C- grade. 

# CRS is the number of course that gave out a grade of C-. 

AVG Sec is the average section size of all section that had C- grades. 

Sec. Unit is the sum of each the impact of each C- student on each course. The impact was calculated by dividing 
the number of students who received a C- in a course by the section size of that course. 



Impact of C- and Course Waitlist on Course Availability (in Section Units) 



Total Section Units by Colleges 





Fall 03 


Spring 04 


Fall 04 


Spring 05 




C-& 


# 


AVG 


Sec 


C-& 


# 


AVG 


Sec 


C-& 


# 


AVG 


Sec 


C-& 


# 


AVG 


Sec 


College 


Wait 


Crs 


Sec 


Unit 


Wait 


Crs 


Sec 


Unit 


Wait 


Crs 


Sec 


Unit 


Wait 


Crs 


Sec 


Unit 


A 


10 


5 


30.0 


0.3 


30 


1 


40.0 


0.8 


25 


5 


35.0 


0.6 


10 


3 


30.0 


0.3 


B 


25 


5 


45.0 


0.6 


25 


3 


40.0 


0.6 


35 


5 


50.0 


1.7 


— 


— 


— 


— 


C 


1300 


125 


30.0 


43.3 


1150 


100 


30.0 


38.3 


1300 


125 


30.0 


51.1 


1375 


135 


25.0 


55.0 


D 


300 


25 


35.0 


8.6 


200 


25 


35.0 


5.7 


250 


25 


35.0 


7.0 


150 


20 


35.0 


4.3 


E 


1200 


75 


50.0 


24.0 


1100 


80 


50.0 


22.0 


1100 


75 


50.0 


29.1 


1000 


75 


55.0 


18.2 


F 


275 


25 


50.0 


6.9 


250 


25 


40.0 


6.3 


275 


25 


35.0 


12.1 


225 


20 


40.0 


5.6 


G 


400 


30 


40.0 


10.0 


350 


35 


35.0 


10.0 


400 


35 


35.0 


14.9 


300 


30 


35.0 


8.6 


H 


50 


5 


40.0 


1.3 


75 


10 


35.0 


2.1 


50 


10 


35.0 


1.7 


100 


10 


35.0 


2.9 


I 


175 


25 


40.0 


4.4 


150 


25 


30.0 


5.0 


200 


25 


35.0 


8.9 


175 


25 


35.0 


5.0 


J 


175 


20 


50.0 


3.5 


200 


20 


50.0 


4.0 


225 


25 


45.0 


5.6 


250 


25 


45.0 


5.6 


K 


20 


5 


30.0 


0.7 


40 


10 


40.0 


1.0 


25 


10 


35.0 


1.1 


10 


5 


40.0 


0.3 


L 


0 


1 


25.0 


0.0 


20 


5 


35.0 


0.571 


10 


10 


25.0 


0.536 


10 


5 


30.0 


0.333 


TOTAL 

(ALL) 


3930 


346 


37.9 


103.5 


3590 


339 


38.3 


96.4 


3895 


375 


37.1 


134.4 


3605 


353 


36.8 


106.0 



C- & Wait is the number of students that received a C- grade combined with the waitlist. 

# CRS is the number of course with a waitlist that gave out a grade of C-. 

AVG Sec is the average section size of all sections. 

Sec. Unit is the sum of each the impact of each C- plus waitlist on each course. The impact was calculated by 
dividing the number of students. 
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Abstract: Annual performance reporting from Pennsylvania State System of Higher Education ’s 
(PASSHE ’s) System Accountability Program includes qualitative reporting and evaluations of 
the Narrative Assessment Statements and University Performance Plans. The 14 state-owned 
universities provide actions and outcomes in support of five Strategic Plan Goal Categories 
identified in the PASSHE ’s Strategic Plan. Each university submission is evaluated according to 
six criteria including “lessons learned. ” Qualitative performance reporting is linked with the 
Strategic Plan Goal Categories to move PASSHE and its universities in strategically desirable 
directions. 



Introduction 

Since 2001-02, the Pennsylvania State System of Higher Education (PASSHE) has 
published the annual System Accountability Report (Report) on institutional perfonnance for the 
14 universities separately and together as a System (Armstrong, et. ah, 2004). A key rationale 
for producing performance-based reports is that higher education institutions are increasingly 
expected to provide evidence of accountability to stakeholders, including boards of trustees, 
accrediting agencies, legislators, students and families (Borden and Banta 1994). 

As is the case with the PASSHE Report, measurements of performance usually include 
quantitative and/or qualitative data used to describe and assess an institution as it pursues its 
goals (Borden & Bottrill, Summer 1994). Furthennore, reporting is not confined to an 
assessment of the university only; value-added can be obtained from sharing lessons learned 
across universities that can enhance beneficial outcomes. 

This year’s Report (August 2006) analyzed performance for 17 quantitative measures and 
additional qualitative measures. Each quantitative and qualitative measure is aligned with 
PASSHE’s Strategic Plan Goals. According to a recent survey by Burke and Minassians (2003), 



’The authors would like to recognize Jeff Kinsey, Sara Senko, Scott Souders, Melinda Tobin, and Savita Wani, 
Pennsylvania State System of Higher Education, for their contributions to the System Accountability Program. Any 
errors are the responsibility of the authors. 
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46 states, including Pennsylvania, have implemented some form of accountability reporting for 
higher education. 

Performance reporting is not static; reports change as stakeholders require different 
summary analyses. This year’s Report (August 2006) provided an expanded qualitative 
evaluative framework for the actions/outcomes of the Narrative Assessment Statement (NAS) 
and University Perfonnance Plan (UPP) narratives and was included within the Executive 
Summary (see Armstrong, et. ah, 2004). Furthermore, the Report included a single Matrix 
Narrative Report, Tab 2, which satisfied the reporting needs of both the NAS and UPP of all the 
universities. In addition, a summary Report (October 2006) was produced that included lessons 
learned or highlights from the content evaluation of the narratives from all 14 universities. The 
next section provides details of the single Matrix Narrative Report within the System 
Accountability Program. 

Submissions Methodology for the Matrix Narrative Report 

PASSHE developed the System Accountability Program (SAP) to assess the overall 
performance level of the System and each university on an annual basis. The first System 
Accountability Report was published in 2001-02. In 2004, the Board of Governor’s adopted 
PASSHE’s Strategic Plan, Leading the Way, which resulted in the alignment of quantitative and 
qualitative measures within Strategic Plan goal categories. PASSHE’s Strategic Plan goal 
categories are (1) Student Achievement and Success, (2) University and System Excellence, (3) 
Commonwealth Service, (4) Resource Development and Stewardship, and (5) Public Leadership. 

Within each goal category, universities report narratives that link to either the NAS 
and/or the UPP. The NAS focuses on performance results that are not easily measured 
quantitatively. The UPP are university-specific goals and initiatives that reflect the strategic 
direction of the university and are designed to facilitate the measurement of the university’s 
performance. The UPP focus is on measurable perfonnance results that are either quantitatively 
or qualitatively described. The NAS and UPP reporting categories are organized under the 
appropriate goal category, ensuring an integration of strategic planning, reporting, and the 
evaluation of university perfonnance outcomes. 

There are four required NAS reporting categories or “NAS descriptors”: (1) accreditation, 
(2) teacher certification tests, (3) system partnerships, and (4) private giving and endowments. 

All other reporting descriptors are optional. 

University responses to UPP reporting categories are also presented under each Goal 
Category. The UPP reporting categories or “UPP goals” allow institutions more flexibility in 
highlighting university goals and showing evidence of progress in achieving them. There are 16 
UPP goals that are aligned with each Strategic Plan Goal Category and also shown in Table 1. 
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Table 1: Strategic Plan Goal Categories and UPP Goals* 



Student Achievement and Success 


Resource Development and Stewardship 


Manage Growth/Quality 


Effective Use of Resources 


Quality Instruction, Resources, and Support 


Alternative Funding 


Leadership and Life-long Learning 


System Technology Consortium (SyTEC) 


University and System Excellence 


Commonwealth Service 


Quality Academic Programs 


Commonwealth Programs 


Diversity and Excellence 


Regional Economic Development 


Development of Faculty, Administrators, and Staff 


Teacher Preparation 


Continuous Improvement 


Graduate Programs 


Public Leadership 


Policy Advocate 


Advance System Vision 



*The UPP goals are aligned with each bolded Strategic Plan Goal. 
Source: Leading the Way (2004). 



The narrative reporting structure is organized around each institution’s own strategic plan 
or planning document. The number of goals generally ranges from 20-40. The goals of the 
University Plan are the starting point for all narrative reporting, but for this report, the narrative 
is organized by each Strategic Plan goal category, and the appropriate NAS descriptor and UPP 
goal narrative are reported under those categories. For each goal of a University Plan, 
universities entered a maximum of four actions taken in the past year towards achieving that 
goal. The basic reporting timeline is the academic year adjusted as June 1, 2005 to May 31, 
2006. 



Under each action, universities identified a maximum of three outcomes for that action. 
Outcomes can be both quantitative and qualitative. Some actions may have only a single 
outcome whereas others will have multiple outcomes. The limit requires the university to select 
the most important outcomes for each of the actions. Actions/ outcomes may be identified as 
multiyear (in progress) or single year. 

The next section summarizes the System-wide Strategic Plan Goal Category qualitative 
reporting outcomes of the System Accountability Report 2005-2006. The System is comprised 
of the 14 PASSHE universities: Bloomsburg, California, Cheyney, Clarion, East Stroudsburg, 
Edinboro, Indiana, Kutztown, Lock Haven, Mansfield, Millersville, Shippensburg, Slippery 
Rock, and West Chester. 

System-Wide Strategic Plan Goal Category Outcomes 

Performance reporting from the System Accountability Program for PASSHE’s 14 
universities includes qualitative and quantitative components to develop an understanding of 
university efforts and accomplishments. The qualitative reporting components are the NAS and 
UPP that report university actions and outcomes in support of System Goal Categories identified 
in PASSHE’s Strategic Plan, Leading the Way. 

Universities provided detailed descriptions of actions and outcomes in each of the five 
Strategic Plan Goal Categories and for each of the 16 Plan Goals. There were a total of 1,898 
outcomes as reported in Table 2. All universities reported outcomes for each goal in Student 



22 





Achievement and Success and University and System Excellence. Additionally, all universities 
reported outcomes for some goals within Commonwealth Service and Resource Development 
and Stewardship. Not surprisingly, the largest number of outcomes was in Student Achievement 
and Success and University and System Excellence. 



Table 2: Number and Frequency of Qualitative Outcomes within each of the Five Strategic Plan 
Goal Categories 





PASSHE Strategic Plan Goal Categories 


University 


(1) Student 
Achievement and 
Success 


(2) University 
and System 
Excellence 


(3) Commonwealth 
Service 


(4) Resource 
Development 
and Stewardship 


(5) Public 
Leadership 


Bloomsburg 


35 . 1 % 


24 . 5 % 


11 . 7 % 


21 . 3 % 


7 . 4 % 


California 


42 . 2 % 


38 . 3 % 


7 . 2 % 


9 . 4 % 


2 . 8 % 


Cheyney 


33 . 9 % 


35 . 6 % 


10 . 2 % 


18 . 6 % 


1 . 7 % 


Clarion 


35 . 5 % 


22 . 6 % 


16 . 1 % 


16 . 1 % 


9 . 7 % 


East 

Stroudsburg 


31 . 1 % 


44 . 7 % 


13 . 6 % 


10 . 7 % 


0 . 0 % 


Edinboro 


16 . 8 % 


37 . 2 % 


11 . 5 % 


22 . 1 % 


12 . 4 % 


Indiana 


39 . 3 % 


24 . 8 % 


17 . 9 % 


17 . 2 % 


0 . 7 % 


Kutztown 


33 . 8 % 


32 . 0 % 


17 . 3 % 


15 . 6 % 


1 . 3 % 


Lock Haven 


37 . 9 % 


29 . 2 % 


20 . 8 % 


10 . 4 % 


1 . 7 % 


Mansfield 


22 . 9 % 


33 . 0 % 


20 . 2 % 


18 . 3 % 


5 . 5 % 


Millersville 


27 . 3 % 


23 . 0 % 


17 . 0 % 


30 . 3 % 


2 . 4 % 


Shippensburg 


26 . 8 % 


35 . 8 % 


23 . 6 % 


13 . 0 % 


0 . 8 % 


Slippery Rock 


36 . 4 % 


30 . 3 % 


12 . 1 % 


15 . 2 % 


6 . 1 % 


West Chester 


33 . 8 % 


33 . 8 % 


19 . 9 % 


10 . 2 % 


2 . 3 % 


Total (1,898) 


625 


601 


313 


297 


62 


Percentage of 
Total 


32 . 9 % 


31 . 7 % 


16 . 5 % 


15 . 6 % 


3 . 3 % 



Source: System Accountability Report - Performance Outcomes 2005-2006 Data Submissions. 



Each university’s submission is evaluated according to the following six criteria: Does it 
support the goal? Is there evidence of this support? Are there measurable results? Is there 
evidence of progress? Does it contribute to performance? Does it provide a model of lessons 
learned (“highlights”) for others? Qualitative evaluation attempts to provide an understanding of 
an institution’s performance, measuring achievement, advancing policy development, and 
providing inference from data received (Spencer, et. ah, 2003, August; Light, 2000, November). 



The evaluation process consisted of two teams of raters from Academic and Student 
Affairs and System Research that evaluated each outcome independently. For all the criteria 
except highlights, an affirmative answer to the criteria included the outcome within the criteria. 
For highlights, if rates disagreed, a third rater evaluated the outcome. The outcome must have at 
least two affirmative answers to be included for this criterion. All universities had at least one 
highlight outcome. 



From the System Accountability Report summary (2006, October), a brief narrative of 
the types of actions and outcomes is reported. In addition, one representative highlight 
(university name is masked) for each Goal Category is provided. 
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1. Student Achievement and Success 

There were 625 outcomes or 32.9% of the total outcomes reported in support of this Goal 

Category, which was the most for any Goal Category. Every University listed outcomes for each 

of these three goals. 

A. Manage growth to ensure access while enhancing the quality of State System 
Universities. 

The actions and outcomes reported for this goal are clustered into 23 broad categories, 
with the most frequently cited outcomes in enrollment management (43%), retention and 
graduation (14%), and community college transfers (8%). These are the areas where 
universities can make the greatest difference for new and returning students. 

B. Enhance the quality of instruction, learning resources and support services 
available to students. 

Continuing its dominance from the prior year, this goal once again had more outcomes 
than any other. Universities reported initiatives and accomplishments in 28 different 
areas. More outcomes were included by far for learning environment (15%), library 
(12%), and technology (11%) than any other category. This distribution is intuitive, as 
the learning environment is perhaps the most important factor contributing to effective 
teaching and learning, while sufficient library resources are critical to support the 
academic enterprise, and technology has become a vital component in integrating efforts 
to enhance classroom and distance education instruction. 

C. Provide all students with opportunities leading to active citizenship, social 
responsibility and life-long learning. 

Most readers would consider the two previous goals — managing growth to ensure access 
and enhancing the quality of instruction — as fundamental to the mission of all System 
Universities. What is equally important, but often overlooked, are the other opportunities 
Universities provide students to become socially responsible citizens committed to life- 
long learning. Universities reported many outcomes for leadership and life-long learning. 

Highlight - Action: Opportunities. Expand opportunities for students of all backgrounds to 
attend the University. 

Outcome: Outreach. The Academic Development Services Department administered the ACT 
101 program to 175 low-income college students who are Pennsylvania residents. The five- 
year retention rate for these students is 84%, with a 78% graduation rate and 23% of 
graduates go on to attend graduate or professional school. The TRIO Student Support 
Services Program has 165 low-income first generation college students participate. The five- 
year retention rate for these students was 82%, with a 70% graduation rate and 31% 
enrollment in graduate or professional schools. 

2. University and System Excellence 

601 outcomes or 31.7% of the total outcomes were reported in support of this Goal Category. 

Every University listed outcomes for each of these four goals. 
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A. Focus the efforts of System Universities on high quality academic programs that 
meet the needs of Pennsylvania and its students. 

To show evidence of high quality academic programs, Universities documented 
outcomes in accreditation, curriculum, and external recognition of program quality. In 
fact, these three categories were responsible for 70% of all outcomes, even though 18 
broad categories were included. 

B. Establish diversity as a cornerstone of excellence and leadership throughout the 
System. 

Diversity has always been a high priority for the System, and University progress in 
increasing diversity is measured by 26 quantitative sub-measures and seven performance- 
funding sub-measures. For this goal, Universities submitted outcomes in 21 areas, with 
the greatest frequency in initiatives for students of color (32%), learning environment 
(19%), and faculty quality (17%). 

C. Provide all faculty, administrators and staff with professional and leadership 
development to enhance performance. 

The largest investment System Universities make is in their personnel. Although 
students are the reason Universities exist, the faculty and staff are the ones responsible 
for making the academic enterprise work. 

D. Support an environment of continuous improvement to ensure efficiency, enhance 
effectiveness and pursue excellence in System programs, services and activities. 

Continuous improvement has been practiced in higher education institutions for several 
decades. Of the many outcomes reported in this area, the largest numbers were reported 
in external recognition/program quality (20%), strengthening management practices 
(13%), and learning environment (13%). 

Highlight - Action: Support for Online Teaching. Recognizing that web-based programming 
presents new challenges related to both teaching and learning, the University has mandated 
that all faculty teaching in web-based programs complete the eCollege certification program. 
In addition, eCollege has provided the University with the opportunity to participate in a 
national study by ETS, the eSIR survey. 

Outcome: LISTSERV. During the spring of 2006, the University GO Listserv was created and 
become operational. This listserv provides a vehicle for disseminating useful resources for 
online teaching and learning, as well as serving as a venue for faculty to share best practices. 

3. Commonwealth Service 

There were 313 outcomes or 16.5% of the total outcomes reported in support of this Goal 
Category. 

A. Develop an array of programs designed to meet best the higher education needs of 
the Commonwealth. 

Long before the development of the System’s current Strategic Plan, PASSHE 
Universities were offering a range of professional programs that produced graduates in 
high need fields. That activity has increased since the plan was adopted in 2004. Eleven 
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Universities reported activities in primary areas for this goal. Of the 17 broad categories, 
the most common were enrollment management (14%), science and technology programs 
(11%), and initiatives for students from low socio-economic backgrounds (11%). 

B. Enhance the capacity of the System to serve regional economic and community 
development needs. 

As has been well documented in the recently released Economic Impact Study, System 
Universities have a positive impact on their regions. For this goal, Universities described 
more outcomes in regional economic development (19%), collaboration with business 
and industry (16%) and workforce development (15%) than any of the other 20 
categories. Because many of the same regional economic activities were listed in 
different categories, this summary will describe them interchangeably. 

C. Retain the System’s status as the premier provider of teachers to the 
Commonwealth. 

PAS SHE has always been known for educating large numbers of highly qualified 
teachers. This goal requires Universities to maintain those high standards as they prepare 
students to teach in classrooms. Of the 17 categories listed for this goal, a majority of 
outcomes were in teacher education programs and teacher certification texts (PRAXIS). 

D. Support graduate programs designed to meet the needs of the Commonwealth 

Although the System primarily serves undergraduate students (who make up 87% of all 
enrollments), graduate programs are becoming increasingly important as more 
occupations require graduate training for entry or promotion into technical or leadership 
positions. System Universities listed outcomes in ten general categories, but enrollment 
management, other high-need programs, and teacher education programs each accounted 
for 19% of all responses. 

Highlight - Action: Science and Technology. Continue to develop innovative programs in the 
area of science and technology to meet the rapidly changing needs of society. 

Outcome: Nanotechnology Manufacturing. Seven System Universities were awarded a $200,000 
grant from the Pennsylvania Department of Community and Economic Development to 
develop nanotechnology modules for inclusion into science courses. 

4. Resource Development and Stewardship 

There were 297 outcomes or 15.6% of the total outcomes reported in support of this Goal 
Category. 

A. Ensure that all System resources are used effectively and efficiently. 

The PASSHE Universities are state-owned, which means that a substantial percentage of 
resources come from tax revenues. Therefore, the System has a responsibility to 
legislators, taxpayers, and parents to use these resources effectively and efficiently. 
Practicing wise stewardship is a System value, and one purpose of the accountability 
measures is to detennine how effective and efficient Universities operate (both fiscally 
and programmatically). Although examples of University efforts in meeting this goal 
were found in 24 broad categories, the most commonly cited were initiatives to reduce 
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costs (17%), increase productivity (15%), administratively streamline (11%), and find 
new or expanded revenue sources (1 1%). 

B. Increase the level of alternative funding to support new and existing programs and 
services. 

Even though the State System receives $445 million annually in appropriations, that 
amount represents only 38% of the total $1.2 billion budget. Tuition covers most of the 
remaining expenses, but additional revenues are needed to support new and existing 
programs and services. Universities have been successful in finding additional sources to 
supplement traditional revenues, including private giving efforts, grants and contracts. 

Of the 12 broad categories of outcomes Universities included for this goal, the most 
common were private giving and endowment growth (39%), new or expanded revenue 
sources (29%), and state or federal grants or contracts (22%). 

C, Employ the System Technology Consortium (SyTEC) to promote efficiency and 
effectiveness across the System. 

PAS SHE is implementing a comprehensive enterprise reporting system that includes 
finance, human resources, and student modules (called Campus Management). Now that 
the Finance and Human Resource modules are fully implemented and used by all 
Universities, attention has turned to implementation of Campus Management for the pilot 
institutions. For this goal, Universities listed outcomes in seven categories, with 
technology (30%), administrative streamlining (22%), and increasing productivity (22%) 
responsible for almost three in four responses. 

Highlight - Action: Supplier Diversity Initiative. Continue to implement the Supplier Diversity 
Initiative to expand the University’s vendor base to include all qualified vendors and to 
enhance cost-saving opportunities for the University. 

Outcome: Minority and Women Business Enterprise. University’s minority business 

enterprise/women business enterprise participation in the procurement area increased fonn 
7.8% in 2004-05 to 12.5% in 2005-06. 

5. Public Leadership 

There were 62 outcomes or 3.3% of the total outcomes reported in support of this Goal 

Category. 

A. Shape the policy framework for public higher education in the Commonwealth. 

Although Universities reported outcomes in 16 categories, the most common areas 
reported were collaboration with government (15%) and corporate alliances (13%). An 
excellent example of the type of leadership Universities are practicing is the Government 
Agency Coordination Office (GACO) at California University. This is an economic 
development program that assists businesses with federal, state, and local contracts. 
Presidents and other campus officials are often speakers at events where business and 
community leaders are present, which gives these campus leaders forums for presenting 
information about the State System to influential audiences. 
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B. Advance the vision for the System through the policies, actions, communications, 
and programs of the Universities, the Board of Governors and the Office of the 
Chancellor. 

Universities reported outcomes in ten areas of advancing the System vision. Of these, 
new or expanded revenue sources and learning environment were responsible for 46% of 
all outcomes. Many outcomes were similar to those included in the previous goal. For 
example, Universities reported that their alumni had breakfast meetings with legislators 
and supported System appropriations through letter writing campaigns. University 
presidents are involved in many activities that advance the vision of the System. 

Highlight - Action: Service Learning. Support service-learning activities and student service- 
oriented activities. 

Outcome: Service Activities. Student organizations and athletic teams volunteered over 13,500 
hours in community service projects, including clean-up efforts and other Hurricane Katrina- 
related activities. In addition, the faculty responded to meet the needs of students displaced 
by Hurricane Katrina by providing 4 free online courses through the Sloan-C Foundation. 
The University was one of 200 colleges nationwide and the only PASSHE institution 
involved in this project. 



Conclusion 

Perfonnance reporting is anchored in results management. Strategic management li nk s 
performance reporting with strategic goals of an institution. This year’s System Accountability 
Report 2005-2006 (August 2006) links PASSHE goals with qualitative evaluation assessment of 
university actions and outcomes of their NAS and UPP narratives, including lessons learned, for 
continuous improvement. Universities and stakeholders are benefited by the knowledge they 
gain from this performance-based accountability program. 
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Introduction 

Over a two-year period beginning in March 2004, community colleges in Maryland 
developed a revised set of accountability indicators for state -mandated perfonnance 
accountability reporting based on an improved model for assessing student degree progress. This 
paper describes the components of this new Degree Progress Model, and then examines 
significant patterns in the cross-institutional database containing the same outcome indicators 
supplied to the state, but augmented with demographic and other institutional context data. 
Specifically, the analysis attempts to capture the ways three sets of environmental factors — 
service area characteristics, institutional characteristics, and student body characteristics — 
correlated with graduation-transfer and related outcomes rates for the sixteen Maryland 
community colleges. The analysis is exploratory with several methodological limitations, but 
suggestive of how environmental factors influence student outcomes. The paper concludes with 
a comparison of actual rates to expected rates following the assessment framework advocated by 
Astin. 



Purpose and General Characteristics of the Degree Progress Analysis Model 

The Degree Progress Analysis is a four-year cohort-based model for more realistic and 
useful assessment of community college institutional performance. The model was developed by 
a taskforce of the Maryland Community College Research Group (MCCRG), an affinity 
organization of Maryland community college institutional researchers, and has been adopted in 
2005 by the Maryland Higher Education Commission (MHEC) for the student outcomes section 
of its annual community college institutional assessment report. The model takes into 
consideration not only the final outcome rate of program completion, but also transfer rate — 
especially transfer without award - which increasingly represents the main “formal success 
experience” of community college students according to national data. 

The model, however, additionally incorporates several interim measures of pre- 
graduation/transfer success. The most important of these are sophomore status achievement (no 
degree or transfer but 30 or more credits earned and the maintenance of a 2.0+ GPA) and 
continuing enrollment (plus the maintenance of a 2.0+ GPA) at the four-year census point 
without yet having achieved a degree or transfer. The first is a rough indicator of general 
education course completion and the attainment of the halfway point toward program 
completion. The second represents long-tenn retention and persistence in the pursuit of a higher 
education award. The total package of indicators is named the Degree Progress model because 
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it includes multiple measurements of progress toward or attainment of an associate degree 
(sophomore status, continuing enrollment, degree earning), or of progress toward a baccalaureate 
award (transfer rate). This provides a more nuanced and realistic picture of student outcomes, 
allowing for the tracking of types and relative levels of academic achievement. Furthermore, as 
a data analysis found in this paper will illustrate, the inclusion of interim indicators has the added 
benefit of providing institutional researchers and college administrators with ways of identifying 
trouble spots within the academic process that may spoil student chances for academic success. 
This moves institutional performance assessment away from its current bottom-line fixation and 
towards utility as a diagnostic tool. 

As already mentioned, the indicator data of Degree Progress model is the result of cohort 
fourth-year analysis. The cohort employed in our model cohorts diverges from the conventional 
IPEDS definition (degree-seeking, full-time, first-time anywhere fall entrants). Model cohorts 
consist of all new fall entrants, without respect to their semester credit load or degree-seeking 
status. The only additional requirement for cohort inclusion is that its students have a record of 
18 attempted credit hours. This approach was inspired by the cohort-defining criteria developed 
by the state of Florida for college performance assessment. Florida authorities were attempting 
to avoid the notoriously problematic identification of “degree-seekers” by means of highly 
volatile program curricula choices or admission-time response to an academic goal question. It 
was their feeling that students who attempted at least 1 8 credit hours had behaviorally expressed 
a seriousness of academic purpose tantamount to objective degree-seeking. They also reasoned 
that limiting cohorts to full-time students in the community college case tended to exclude the 
majority of those enrolled, leading to unrealistic outcomes results. Here also the 18 credit hour 
rule would prove helpful as a sort of full-time/part-time student equalizer in a shared cohort. The 
Degree Progress model as adopted by MHEC included sub-cohort reporting by developmental 
status groups (initially, college-ready according to developmental placement results; 
developmental program completers; and developmental program non-completers) and by 
race/ethnicity categories. (See Boughan and Clagett (2006) for a detailed review of model’s 
methodological underpinnings.) 

The Degree Progress Model Illustrated 

Table 1 presents the full Maryland Community College Degree Progress Model with 
example data drawn from Carroll Community College’s 2000 cohort. Currently, the model has 
nineteen interlocking components. The first three define the reporting cohort and the remainder 
provides the actual assessment data based on the performance of students in the reporting cohort, 
during the four-year assessment interval. The table’s line definitions show how each component 
rate is to be calculated and used in the construction of higher level components. The two shaded 
rows indicate the two perfonnance assessment rates — percent graduating or transferring, and 
percent successful or persisting — that the MHEC has approved and will be requiring as part of 
the state’s community college accountability reporting beginning this year. 
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Table 1. The Elements of the Maryland Community College Degree Progress Paradigm 


Line 


Line Definition 


Carroll Community 
College Example 


1 


First-time-any-college fall headcount 


634 


2 


Attempting fewer than 1 8 hours over first 2 years 


204 


3 


Cohort for analysis (Line 1 - Line 2) 


430 


4 


% Earned Associate degree from this community college 


26.7% 


5 


% Earned certificate, but no degree, from this community college 


1.2% 


6 


% Total Associate and certificate graduates (Line 4 + Line 5) 


27.9% 


7 


% Transferred to Maryland two-year/technical college 


6.3% 


8 


% Transferred to Maryland public four-year college 


24.2% 


9 


% Transferred to Maryland private 4-year college or university 


4.7% 


10 


% Transferred to out-of-state college or university (lOa+lOb) 


10.9% 


11 


% Total transfers (unduplicated sum of Lines 7-10) 


46.0% 


14 


% Graduated this college and transferred (Line 6 | [ Line 11) 


19.5% 


15 


% Graduated and/or transferred {(Line 6 + Line 1 1) - Line 14} 


54.4% 


16 


% No award or transfer, but 30 credits with GPA > 2.00 


16.0% 


17 


% Successful transition to higher ed (Line 15 + Line 16) 


70.5% 


18 


% Enrolled at orig. institution last term of study period (Continuing) 


3.7% 


19 


% Successful or persisting (Line 17 + Line 18) 


74.2% 


NOTE: Shaded rows represent state required accountability measures. 



Originally, the Degree Progress Model included Lines 12 and 13, which represented 
degree progress at the transfer receiving school. At a recent meeting, the MCCRG, charged with 
overseeing the model’s technical development, voted to drop these elements because they do not 
relate directly to state community college outcomes, and because proper assessment would 
require adding at least two years to the assessment interval. It also decided to split Line 10 in 
future reporting into two components — transfer to two-year and to four-year out-of-state schools 
— to allow for the calculation of general two- and four-year transfer rates. (For a full cross- 
school review of all the rates found in Table 1, contact the author for the Appendix.) 

Cross-College Analysis of Fall 2000 Degree Progress Data After Four Years 

Research Context. In the remainder of this paper, we will present an analysis of 
community college performance data based on the Degree Progress reports collected from all 
sixteen public two-year institutions in the state of Maryland. Our goal was two-fold. First, we 
wanted to gain a deeper understanding of the full range of cross-institutional student outcome 
phenomena, in how the different schools compared and contrasted on a variety of indicators and 
what this suggested about the dynamics of their student bodies. The second goal was to trace as 
much of the above variation, especially that of the key graduation/transfer rate indicator, back to 
differences in school demographic environment, institutional characteristics, and type of student 
body. 



Two important issues in educational assessment methodology suggested these research 
objectives. The first deals with questions concerning the utility of standard outcomes measures: 
How adequate is the classic graduation rate indicator (or graduation-transfer rate indicator) as a 
general measure of institutional performance? Does it do a good enough job of expressing the 
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relative academic success of its student body? Is it helpful in identifying those problem points in 
an academic process that needs addressing? Astin’s (1993a) famous I-E-0 model of institutional 
performance posits a critical intervening “environmental” component between institutional input 
and output. Does this not suggest the utility of employing interim measures of student progress 
as supplements to final outcomes indicators? 

The second issue cluster involves questions concerning the validity of assessing 
institutional performance without consideration of environmental context: How fair is it to judge 
the relative performances of community colleges without taking into account, systematically if 
possible, environmental factors importantly conditioning student success probabilities and over 
which institutions may have little or no control? Should not schools be evaluated in tenns of 
what can reasonably be expected given their demographic, financial, and other impacting 
circumstances? This approach, in fact, has been explicitly recommended by Astin (1993b). 

Literature Review. We began our study with a search for past institutional research 
bearing on the environmental correlates of community college student outcomes, especially 
within the context of a state system. We could find only one study that attempted this through a 
systematic analysis. Windham and Hackett (1996) analyzed the retention and graduation 
variation across Florida’s 28 two-year public colleges by means of linear regression analysis and 
found several explanatory institutional characteristics: influencing negatively were college size 
(FTE) and predominance of career of degree programs, while the most robust positive predictors 
turned out to be faculty salary and student support expenditure levels. The New Jersey State 
Commission on Higher Education, in a recent report (1998) detailed graduation and transfer 
trends for New Jersey’s 19 community colleges but carried out systematic analysis of student 
outcomes only on four-year public school data. This study was noteworthy, however, for using 
the case residuals (actual - predicted outcome) of its institutional characteristics-based predictive 
equation to measure “true” level of institutional performance. Neither of these two studies 
employed transfer as a student success indicator, nor did they consider service area 
demographics as an institutional background category. 

A larger set of studies featured national community college databases built out of IPEDS 
and national survey data (e.g. NCES longitudinal surveys, CIRP, the Community College 
Student Experience Survey), but differed significantly in their selection of independent and 
dependent variables. For example, Bailey, Calcagno, Jenkins, Kienzl, and Leinbach (2005) used 
hard institutional characteristics data such as school size, proportion student body minority, and 
part-time student and part-time faculty percentages, finding that all of these correlated negatively 
with dependent variables measuring retention and degree attainment. They also discovered that 
unit level student characteristics tended to be stronger detenninants of student outcomes than 
institutional characteristics in their equations. Habley and McClanahan (2005), on the other 
hand, defined institutional characteristics in tenns of student support service configurations and 
reported a list of best practices, which associated strongly with institutional graduation rates, 
including the presence of various types of tutoring centers, advisor intervention programs, and 
ethnic support groups. Glover and Murrell (1998) took still another tack, basing their 
institutional characteristic data on student perceptions of peer and staff friendliness, faculty 
accessibility and course intellectual stimulation, and their outcomes variables on respondent- 
reported satisfaction with academic career and sense of intellectual and personal growth. These 
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two sets of factors were found to be significantly correlated. Then there is the study of Alfred 
and Peterson (1992) who conceptualized the characteristics of institutions in terms of dimensions 
of structure and operation. Their survey of a national sample of community college 
administrators and faculty uncovered a strong link between student success and attendance at 
schools featuring a “responsive” organizational culture known by its concern for school 
reputation in the student body and larger community, innovativeness and flexibility in the 
provision of student support services, and the active promotion of student- faculty/staff 
interaction. Last, there is the work of Yang (2005), noteworthy as the only institutional 
environment research we could find using transfer attainment as its dependent variable. Yang’s 
study related transfer rate negatively to revenue/FTE, high tuition, and the percentage of part- 
time and minority attendees in the student body. None of the studies, however, paid any 
attention to the possible impact of community college service area characteristics on student 
outcome, and none (with the exception of Yang’s) saw fit to include transfer attainment in their 
measures of student success. 

Finally, we ran across a fair number of four-year school studies exploring the connection 
of institutional characteristics and graduation, most notably those by Astin, Tsui and Avalos 
(1996), Goenner and Snaith (2004), Mortenson (1997), Pascarella and Terenzini (1991), Porter 
(2000), and Scott and others (2004). The positive success correlates commonly turned up were 
high national test scores; good high school GPA; and student bodies with concentrations of full- 
time, female, minority, and lower aged attendees. They also generally agreed on the negative 
impact of school size, location in an urban environment, disproportion of part-time faculty, low 
spending on student services and general under funding. 

Study Methodology. The plan for analyzing the Maryland degree progress data had two 
parts. The research objective of the first part was to explore what the patterns in our expanded 
set of student outcomes indicators might tell us about the dynamics of student flow through 
Maryland’s 16 community colleges. Was it fairly uniform or were there interesting variations of 
interim and final outcomes to be detected? The analysis mainly involved examination of special 
arrangements of selected key indicators designed to bring out different outcomes “styles” if any. 
Outcome data were then subjected to systematic treatment by means of a cluster analysis 
(Ward’s method) to find discrete outcome pattern groupings among institutions. The data for 
analysis of outcomes by student developmental status was short one institutional case, the 
Community College of Baltimore County, due to data processing problems, that in analytic 
instances involving developmental data reduced the active sample to 15. 

In the second phase of the research, we explored why institutional performance varied as 
it did. To what extent could different student outcomes by college be traceable to environmental 
factors? We conceptualized “environment” as three sets of possible explanatory variables — 
service area demographic and socio-economic characteristics; institutional characteristics 
involving level of financial resources, expenditure patterns, and faculty traits; and student body 
demographic and academic characteristics. 

The service area data was derived from the 2000 U.S. Census, conveniently coinciding in 
time with the starting year of our student cohorts. Most Maryland community colleges serve a 
particular county, but three are assigned to multiple county territories, and in the later instance, 
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census data was pooled for these areas. For 2000 data on institutional and student body 
characteristics, we drew mostly from the historical data book published by the Maryland 
Association of Community Colleges (2005). Data included race/ethnicity, gender, age group, 
first semester course load, degree program, and Pell grant receipt. Developmental status came 
straight from the degree progress submissions. 

To eliminate multicollinearity effects, always troublesome in data of this kind, and to 
identify the dimensions underlying the large number of variables we had assembled, we 
subjected the data elements of the three environmental sets to separate factor analyses (least- 
squares measurement, orthogonal rotation). This method yielded 10 meaningful factors 
described latter in this paper. These, normally distributed by mathematical nature, constituted 
the independent variables in a series of multiple linear regression analyses. As an independent 
variable for all of the regressions, we selected the graduation and/or transfer indicator (GT). 

This measure proved to be reasonably nonnal by the usual statistical tests (kurtosis=.493, 
Kohnogorov-Smirnov Z=.638). 

This analysis is exploratory only, and we make no claims that the results of our 
regressions are in any way definitive, always statistically significant, or reliable overall or 
particularly in detail. We are well aware that using linear regression where the sample is slight 
(here n=16 and occasionally 15 cases) and where the data are aggregate involves important 
methodological problems, not the least of them the tendency to obtain very inflated estimates of 
association. Our defense is that the regression approach was the most efficient means to explore 
multiple environmental impacts on student outcomes, and in hindsight, that they yielded results 
making theoretical sense and generally resembled the findings of other researchers working in 
this area. As a practical matter, we feel that our methodology was sufficient to establish broad 
patterns of impact and to point the way to more robust future research. 

Findings of the Degree Progress Data Analysis 

Outcome Patterns. Table 2 displays cross-school summary data for the Degree Progress 
Model’s two accountability indicators. When the 2000 cohort reporting for Maryland’s 16 
community colleges was pooled, we found a rounded total system graduation/transfer (GT) rate 
of 47 percent and a 76 percent “success or persistence” (SP) rate (the graduation/transfer rate 
augmented by sophomore status attainment and continued study rates). When transfer 
attainment, a genuine indicator of educational progress, is joined with graduation attainment into 
a single formal achievement indicator, the resulting measure reveals an encouraging degree of 
academic progress made over four years by Maryland’s recent community college students. 
Almost half of them managed to meet their degree goal or to advance to the four-year level in 
their academic pursuits. Furthermore, when we allow for important pre-graduation achievement 
like attaining sophomore status (tantamount in most cases to fulfilling a student’s general 
education requirements) and retention through the fourth year, we can fairly say that over three 
out of four students made significant progress toward their degrees after four years. 

Lastly, Table 2 calls attention to what institutional researchers have known all along, that 
much of the poor institutional performance typically reported by community colleges results 
from their high remediation burden. Almost three-fifths (59 percent) of Maryland community 
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college students who began their studies ready to begin taking college-level credit courses (no 
placement into developmental programs) managed to graduate or find places at four-year 
institutions within four years. In contrast, only 30 percent of those failing to finish their 
remediation programs did so (their GT rate reflects transfer success only). It also shows that 
developmental completers tended to do almost as well as college-ready students in terms of 
formal academic achievement (GT=53%) and equaled them in general academic progress 
(SP=84%), an indication of the efficacy of the state’s community college developmental 
programs when faithfully worked through. 



Table 2. Maryland Community Colleges (N=16) 

All Student Summary of Key Progress Rates of Fall 2000 Cohort after Four Years 




Graduating and/or Transferring 


Successful or Persistent 


All Students (N=14,588) 


46.8% 


75.6% 


College-ready Students (N=4,974) 


58.6% 


84.0% 


Developmental Completers (N=3,652) 


52.8% 


84.0% 


Developmental Non-Completers (N=3,446) 


29.6% 


53.4% 



The above summary statistics, however, hide a multitude of important underlying 
institutional variations. A cross-school report of key indicator results would show GT values 
ranging from a high of 66 percent (Allegany College of Maryland) to a low of 23 percent 
(Baltimore City Community College), and SP values ranging from 92 to 45 percent (bounded by 
the same schools), with significant differences for interim indicator results. The next three tables 
are designed to bring out such degree progress patterns. Table 3 displays individual college rates 
for each component of the summary Success or Persistence indicator. 



Table 3. 2000-2004 Success or Persistence Variations Across Maryland Community Colleges* 


Community College 


% Not Successful 
or Persisting** 


% Sophomore Status 
(No Award or 
Transfer) 


% Graduating and/or 
Transferring 


% Continuing at 
Original CC 
(Last Term) 


Statewide 


23.0 


20.3 


46.8 


9.9 


Allegany C of MD 


8.0 


24.5 


66.2 


1.3 


Garrett CC 


24.0 


12.8 


63.2 


0.0 


Frederick CC 


24.6 


3.5 


63.1 


8.8 


Howard CC 


25.2 


16.5 


56.2 


2.1 


Harford CC 


24.1 


17.2 


54.5 


4.2 


Carroll CC 


25.9 


16.0 


54.4 


3.7 


Hagerstown CC 


15.3 


19.0 


54.0 


11.7 


C of Southern MD 


10.9 


28.7 


53.8 


6.6 


Cecil CC 


16.3 


23.2 


51.6 


8.9 


Montgomery C 


18.4 


20.4 


50.6 


10.6 


Anne Arundel CC 


22.5 


22.1 


47.8 


7.6 


Wor-Wic CC 


29.9 


20.1 


45.3 


4.7 


CC Baltimore Co 


16.3 


23.2 


43.4 


17.1 


Chesapeake C 


37.3 


21.8 


36.7 


4.2 


Prince George's CC 


29.9 


21.0 


29.1 


20.0 


Baltimore City CC 


55.1 


13.7 


23.0 


8.2 


*Row percentages sum to 100%. **100% - % Successful or Persisting 
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Table rows, representing individual colleges, are arranged in GT rate order (highest to 
lowest). The columns display outcomes in rough student throughput sequence. The first gives 
the percentage of a school cohort failing to fall into any of the SP sub-categories of achievement, 
an indication of the dropout rate. Column two shows the proportion of those without fonnal 
academic achievement, who nevertheless managed to attain sophomore status after four years of 
study. The third column gives the GT rate, while the last shows the percentage of cohort 
students still attending each school in the last term of the assessment period (continuing 
students). 



The table patterns clearly suggest that degree progress is a complex phenomenon. For 
example, while the GT rates of the Allegany College of Maryland and Garrett Community 
College were both very high (66 and 63 percent, respectively), the former’s drop out rate was 
considerably lower (8 compared with 24 percent), and its Sophomore Only category was double 
that of the latter. At the opposite end of the GT spectrum, the GT rates of Prince George's (29 
percent) and Baltimore City Community College (23 percent) are shown to be quite similar, but 
PGCC’s dropout rate turned out to be 25 percent lower, and its Continuing Student rate was 
more that twice BCCC’s rate (20 to 8 percent, respectively). 



The findings displayed on Table 4, which breaks down the GT rate into its discrete 
components (Award Only, Transfer Only, and Both Graduation and Transfer), show a similar 
variety of outcomes. At the statewide level, the majority of fonnal achievement turned out to be 
transfer without degree (54 percent), but at Prince George's Community College, 82 percent of 



Table 4. 2000-2004 Pattern of Formal Achievement Across Maryland Community Colleges 



Community Colleges 


% Cohort Award 
and/or Transfer 


% Formal Achievers* 


Award Only 


Award + Transfer 


Transfer Only 


Statewide 


46.8 


22.2 


23.9 


53.8 


Allegany C of MD 


66.2 


78.9 


12.5 


8.6 


Garrett CC 


63.2 


26.3 


27.2 


46.5 


Frederick CC 


63.1 


20.9 


29.0 


50.1 


Floward CC 


56.2 


8.4 


16.5 


75.1 


Harford CC 


54.5 


15.4 


35.6 


49.0 


Carroll CC 


54.4 


15.4 


35.8 


48.7 


Hagerstown CC 


54.0 


21.3 


39.6 


39.1 


C of Southern MD 


53.8 


12.8 


19.0 


68.2 


Cecil CC 


51.6 


24.6 


17.2 


58.1 


Montgomery C 


50.6 


12.1 


19.4 


68.6 


Anne Arundel CC 


47.8 


18.2 


29.5 


52.3 


Wor-Wic CC 


45.3 


31.3 


20.8 


47.9 


CC Baltimore Co 


43.4 


15.2 


27.0 


57.8 


Chesapeake C 


36.7 


33.0 


14.2 


52.9 


Prince George's CC 


29.1 


5.8 


12.0 


82.1 


Baltimore City CC 


23.0 


12.6 


9.6 


77.8 



*Formal Achiever row percentages sum to 100%; column percentages are inferred from the calculation of 
% Award + Transfer (% Awards + % Transfers) - % Graduated or Transferred. 



successful students fell into this category, while only 9 percent of Allegany College of 
Maryland’s cohort successes did so. ACM’s predominant type of success was the terminal 
degree (79 percent) compared with an all-college 22 percent. On the other hand, the modal 
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success category for Hagerstown Community College’s high performing students was the 
traditional graduation and transfer outcome — 40 percent, a rate nearly double that of the 
statewide level. We may also note that the table’s figures provide a partial explanation of the 
low graduation rates found for schools like Prince George's, Baltimore City, and Howard 
Community Colleges; transfer-only students made up 75 percent or more of their student 
successes. 

Given the great diversity of school outcomes portrayed in Tables 3 and 4, we decided to 
perform a cluster analysis of the indicator data to clarify the patterning of the cross-school 
results. Table 5 presents the product of a Ward hierarchical grouping analysis of the Table 3-4 
data. The analysis yielded six discrete clusters defining the Degree Progress personalities of the 



Table 5. 2000-2004 Maryland Community College Degree Progress Clusters 
with Cluster Variable Indexed Means* 








Degree Progress Clusters 








#1 


#2 


#3 


#4 


#5 


#6 




Degree Progress 
Indicators 


Career 

Student 

Success 

(n=l) 


Traditional 

Student 

Outcomes 

(n=5) 


Good 

General 

Perf 

(n=6) 


Career 

Student 

Muddle- 

Through 

(n=2) 


Delayed 

Student 

Progress 

(n=l) 


Struggling 

Unprepared 

(n=l) 


All- 

School 

Indicator 

Rate 


Basic Success Components 
Not Successful or 
Persisting 
Sophomores Only 
Graduating and/or 
Transferring 
Continuing at Original 
CC 

Successful or Persisting 


33 

129 

134 

17 

121 


95 

72 

117 

76 

102 


76 

118 

102 

118 

108 


140 

110 

83 

59 

87 


176 

111 

59 

267 

76 


230 

72 

46 

109 

59 


24.0% 

19.0% 

49.6% 

7.5% 

76.0% 


Type of Formal Achievemen 

Graduating Only 


<t 

447 


99 


65 


113 


15 


25 


11.7% 


Graduating and 


70 


163 


91 


62 


30 


19 


11.8% 


Transferring 




Transferring Only 


22 


104 


124 


79 


92 


69 


26.1% 


Supplementary Student Date 

Full-Time Students 


7 

171 


109 


101 


81 


74 


97 


33.0% 


College Ready Students 


136 


77 


115 


45 


84 


30 


34.1% 


Transfer Program 


75 


136 


92 


72 


94 


49 


44.2% 


Students 














*Indexed Mean = 100 * (Mean Cluster % / All-School %) 










Cluster Colleges: 


#1 Allegany; #2 Carroll, Frederick, Garrett, Flagerstown, Flarford; #3 Anne Arundel, Cecil, Baltimore County, 


College of Southern Maryland, Floward, Montgomery; #4 Chesapeake, Wor-Wic; #5 Prince George's; #6 


Baltimore City 



16 colleges. The character of each cluster is revealed by the pattern of cluster means for the 
whole set of discriminant variables entered into the analysis. To highlight the inferential weight 
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of the cluster means, these are expressed in the table as indexed means — proportionate 
deviations from the all-school rate (indexed mean = (cluster mean / all school mean) * 100). A 
cluster Index value of 100 indicates that the cluster’s mean equals the all-school mean, a value of 
more than 100 show the proportionate exceeding of the general mean, and one below 100 reveals 
the proportionate collective mean short-fall. Thus, Cluster l’s Sophomore Status index value of 
129 indicates a grouping mean 129 percent that of the all-school average of 19 percent. For post- 
analysis reference in the process of cluster interpretation, Table 5 also includes supplementary 
statistics on college student body academic characteristics. 

In the table, the six clusters are arranged in collective GT rate sequence, from high to 
low. Three were above average performing groups. Cluster 1, nicknamed “Career Student 
Success,” consisted of a single school (Allegany College of Maryland) and was characterized by 
extreme rates of GT success, Sophomore Only attainment, but above all tenninal degree 
acquisition (447 indexed mean). Its student body had very below average proportions of transfer 
program students, and greatly disproportionate concentration of full-time and college-ready 
students. Allegany has the most rural of all Maryland community college service areas. In the 
second cluster (“Traditional Student Outcomes”), the prime formal outcome turned out to be 
both graduating and transferring (163 indexed mean); the associated distinguishing student body 
traits were high enrollment in transfer degree programs, but a relatively low rate of college- 
readiness. Cluster 2 included five schools (Carroll, Frederick, Garrett, Hagerstown, and Harford 
community colleges), mostly middle size schools serving rural but rapidly developing counties. 
The third and most populous cluster, “Good General Performance,” consisted of six colleges, 
mostly with middling to large enrollments in suburban jurisdictions (Anne Arundel, Cecil, 
Baltimore County, Howard community colleges, Montgomery College, and the College of 
Southern Maryland). The cohorts of these schools show above average tendencies to transfer 
before graduating or to persist into the fourth year. Cluster 3 is the only group besides Cluster 1 
to register above state level concentrations of college-ready students. 

Cluster 4 (“Career Student Muddle-Through”) is the first of three with sub-par 
performance levels and includes two rural colleges — Chesapeake and Wor-Wic community 
colleges. Their cohorts had large proportions of unsuccessful students (140 SP indexed rate), 
and if successful, students who tended disproportionately to select the terminal degree path. 
Cluster 4 cohorts also exhibited low rates of full-time attendance, college-readiness, and transfer 
program enrollment. The last two clusters were single-member and decidedly low performing. 
The Prince George's Community College cluster (5) featured students with very high dropout 
and low fonnal achievement probabilities, whose successes were measured mostly in terms of 
pre-graduate transfer. However, what really distinguished the Cluster 5 were its extreme 
disproportion of continuing students (267-indexed mean) and its somewhat high concentration of 
Sophomore Only attainers (111). These traits, along with academic status measures indicating a 
mainly part-time student body with great remediation needs, suggest the existence of a “Delayed 
Student Progress” syndrome, where rate of degree progress is inhibited. Indeed, historical data 
for PGCC indicates that the bulk of formal achievement, especially degree earning, occurs in the 
fifth and sixth year of study. Finally, the analysis identified a sixth cluster consisting entirely of 
Baltimore City Community College, which we nicknamed “Struggling Unprepareds.” Like 
Cluster 5, the dropout rate was extreme, but unlike PGCC, which at least showed a near average 
transfer without degree rate (92-indexed mean) and a very prominent continuing student rate 
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(267), all of BCCC’s achievement and retention levels fell far below the all-college levels. 
Furthermore, student remediation need here was the greatest of any cluster (30 indexed college- 
readinesses). 

The specific make-up and traits of Maryland Community College Degree Progress 
clusters is unlikely to be of great interest to institutional researchers outside our state, but we 
have supplied cluster result details to illustrate an important point of concern to all who work 
with assessment indicators. Establishing an institutional performance hierarchy based on 
graduation rate, or even an enhanced final outcome indicator like Graduation and/or Transfer 
rate, is of little utility, except to those playing the praise and blame game. Assessment indicators 
should really point the way to understanding where degree progress problems occur in an 
institution and how they may be addressed. An assessment indicator system should be useful 
diagnostically, and to perform this function, multiple final and salient interim outcome indicators 
need to be present. Our phase, “one research,” demonstrates that among community colleges, 
degree progress may occur in a variety of stable, distinctive patterns, perhaps each requiring a 
difference ameliorative approach. 

Environmental Correlates of Degree Progress. In the second phase of this research, we 
addressed the question of environment influences upon student academic advancement. 
Specifically, we sought to identify the forces, which facilitate or constrain degree progress across 
Maryland’s 16 community colleges. What are they, at what environmental level do they operate, 
what is their joint impact, and to what extent can the effects of any of the prime determinants 
discovered be harnessed or mitigated by impacted institutions? Answers to these questions are 
important in institutional performance assessment because they will tell us the degree to which 
performance-affecting conditions are fixed, and how much institutional effort can reasonably be 
expected to improve collective student performance. 

To explore these matters we assembled a database representing Maryland community 
college environment circumstances at three levels: college service area demographics, 
institutional characteristics, and student body traits. We would have liked to include among our 
institutional data elements organizational culture and academic policies, but these were 
unavailable. The hypothesis is that the particular constellation of demographic and academic 
factors within a student body might establish the range of possible institutional perfonnance. 

As a preliminary to the environmental factor regression analysis, we subjected the 44 
individual data elements in the database to factor analysis. This was done both to reduce the 
large number of variables to a more manageable number of environmental dimensions and to 
eliminate serious problems of multicollinearity. Three separate factor analyses were carried out, 
one for each set of environmental variables, to maintain the integrity of the three-level distinction 
in the resulting factor measures. 

Table 6 reflects the outcome of these factor analyses and serves as a listing of the original 
data elements. It depicts the dimensional measures emerging from each level-based factor 
analysis, the factor loadings of the individual items that collectively provide the factor thematic 
interpretation, the nickname selected for each factor and the total inter-item variance captured by 
its extraction. 
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Table 6. Environmental Factors with Contributing Variable Factor Foadings 


Service Area Factors (90.9%) 


AF1_SES (45.3%) 

% Advance Degree 0.981 

% HH Income $200,000+ 0.965 

% Bachelor's Degree or Higher 0.957 

%HH Income $150,000+ 0.956 

% Upper White Collar Employment 0.912 

% Blue Collar Employment -0.850 

%HH Income $100,000+ 0.835 

% HH Income $75,000+ 0.716 

% Lower White Collar Employment -0.698 

Median Annual HH Income 0.667 

% No High School Diploma -0.645 


AF2_Stable Homes & Neighborhoods (26.2%) 

% Married/Living with Spouse 0.978 

% African American Non-Hispanic -0.888 

Public School Performance Index (0-1) 0.807 

% Current College Attendees -0.764 

Ratio - Growth = Pop. 2000 / Pop. 1990 0.748 

% Families below Poverty Level -0.703 

% Urban -0.606 


AF3_Aging Areas (19.4%) 

Median Age 0.893 

% HH Income Under $25,000 0.719 

% Same Address as in 1995 0.688 


Institutional Factors (78.9%) 


IFljCollege Costs (22.6%) 

$ Typical Cost of Full-Time Study 0.973 

$ Tuition & Fees 0.954 

% Revenue from Tuition/Fees 0.809 


IF3_Educational Resources (19.5%) 

% Full-Time Faculty with Master's or Better 0.963 

% Full-Time Credit Faculty 0.963 

$ Revenue per FTE 2003 0.514 


IF2_College Size (20.4%) 

# Fall 2000 Credit Students 0.941 

# Fall 2000 Credit FTEs 0.930 

Mean Full-Time Faculty Salary 0.598 


IF4_Local Revenue Support (17.3%) 

% Revenue from Local Funds 0.912 

Ratio State to Local Funding -0.905 

Ratio Student Hours/Faculty Hours -0.498 


Student Body Factors (86.3%) 


SFl_Needy Career Students (44.1%) 

% Financial Aid - Any Grants 0.973 

% Financial Aid - Any Aid 0.962 

% Financial Aid - Pell Grants 0.929 

% Career Program Students 0.876 

% Transfer Program Students -0.750 

% Dev. Status - Dev.Non-Completers 0.587 


SF2_Full-Time White Students (27.9%) 

% White 0.855 

% Full-Time 0.854 

% African American -0.839 

% Dev. Status - Dev. Non-Completers -0.666 


SF3_Developmental Completers (14.4%) 

% Dev. Status - College-Ready -0.932 

% Dev. Status - Developmental Completers 0.753 


NOTE: Parenthetical percentages indicate proportionate total inter-item variance capture 



Three service area factors were detected explaining 9 1 percent of all inter- item variance 
at that level. The first was expected — a factor clearly reflecting the collective socio-economic 
status of area residents (income, educational level, and employment level). The second most 
explanatory factor, however, was a surprise. “Stable Homes and Neighborhoods” enveloped 
items, which especially related to quality of family and residential environment. High loading 
variables here had to do with family stability (percent of two-spouse households), public school 
system performance (state test scores), percent currently pursuing college degrees, proportion of 
families below the poverty line, population density (percent in urban areas), and pace of county 
development (1990-2000 population growth). This factor also absorbed the race/ethnicity 
dimension; percent of African American residents had the second highest loading. The final 
service area factor generated was one we called “Aging Areas,” and pointed to a pattern of 
resident aging, address stability, and low income. 

The factor analysis of institutional characteristics produced four-dimensional measures of 
roughly equal inter-item explanatory power. The first gathered up items relating to educational 
cost levels (average price of a semester’s full-time study, tuition and course fee levels, and 
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proportion of college revenue from tuition and fees). The second seemed to gauge institutional 
size from student headcount and FTE angles (it also weakly reflected the level of full-time 
faculty salary). Next, we found a dimensional measure of “Educational Resources.” Items most 
contributing to the emergence of this factor were those typically taken to indicate quality of the 
instructional environment (percent of faculty with advanced degrees, percent of faculty working 
full-time, ratio of college revenue to FTE). Interestingly, the item most associated with 
instructional quality — the classroom student/faculty ratio — fell into the final factor, which 
otherwise seemed to be characterized by gauges of county government revenue support. 

Lastly, the analysis of student body data produced three difficult-to-name factors that 
appeared to distinguish combinations of traits representing types of students, rather than pure 
trait dimensions. The most inter-item explanatory factor was one we ended up calling “Needy 
Career Students”, based on a trait constellation featuring financial aid receipt (particularly of the 
need-based kind), enrollment in a career degree program as opposed to a transfer degree program 
and uncompleted developmental requirements. The second factor, dubbed “Full-Time White 
Students,” as its nickname suggests included white versus African American racial identity in its 
construction, but also tendencies to study full-time and to have fulfilled all developmental 
requirements if any. The last and weakest factor was based on high negative loading for percent 
college-ready and comparably high loading for percent completing required remediation. 

Table 7 provides the zero-order level of association between college graduation or 
transfer rate, our independent variable in the regression analyses to follow, and the ten 
environmental factors just described. The highest GT factors correlate turned out to be Full- 
Time White Students (r 2 =.685) from the student body set, followed closely in association by 
Stable Homes and Neighborhoods (.539) from the service area set, and then Local Revenue 
Support (.378) from the institutional set. Thus, all three environmental levels were represented 
among the top three variance-explaining dimension measures. 



Table 7. Community College Environmental Factors and Graduated or 
Transferred Indicator Association Zero-Order Pearson r Correlations 




r 


r 


SF2 Full-Time White Students 


0.828 


0.685 


AF2 Stable Flomes & Neighborhoods 


0.734 


0.539 


IF4_ Local Revenue Support 


0.615 


0.378 


IF2_ College Size 


-0.400 


0.160 


SFI Needy Career Students 


-0.296 


0.088 


AF3 Aging Areas 


0.243 


0.059 


SF3 Developmental Completers 


-0.173 


0.030 


IF 1_ College Costs 


0.161 


0.026 


IF3 Educational Resources 


0.057 


0.003 


AF1_SES 


0.044 


0.002 



The remaining seven factors fell far below a reasonable benchmarking of important 
impact at r =.300. The only other factor that had a somewhat robust association with GP was 
College Size (.160). Prominently highlighted as a prime predictor of student outcomes in other 



42 





studies, in the Maryland case size proves to be of secondary impact, and is far outweighed in 
impact by the Local Revenue Support factor. 



The bottom three zero-order predictors are also worth commenting on. It has often been 
argued that college costs may affect decisions to persist and that quality of educational resources 
has a relationship with student learning. Yet neither of these as captured seems to be significant 
explanators of college GT rate. Also surprising to us was the very low explanatory power of 
service area SES. Mere affluence of a college’s environment pales into insignificance compared 
with role played by good family and educational support structures in the local community. 

Moving to the multivariate analysis, Table 8 provides a summary of the results of several 
linear multiple regressions of the ten environmental factors upon the graduation or transfer rate 
independent variable. The first three blocked out portions of the table report, the basic findings 
of separate environmental level forced entry regressions, and the last block shows the all-level 
stepwise regression results. Row shadings point out the factors that survived in parallel stepwise 
regressions. Once again, we emphasize that we realize the methodologically problematic nature 
of using multivariate linear regression for small sample aggregate data analysis. Our intention 
here is entirely exploratory, an attempt to gauge in a very preliminary fashion the large contours 
of the pattern of environmental impact on community college performance. We give no 
particular weight to any given statistic, and claim that at most, equation terms and coefficients 
serve to indicate only the identity and ordering of the relative impacts of true predictors of 
student degree progress across Maryland’s 16 community colleges. 



Table 8. Regression Analyses of Environmental Factors upon Graduation or Transfer 




Std Beta 


Sig. 


Part-r 2 


Service Area (R 2 =.600, .500*) 








AF2 Stable Families and Neighborhoods 


0.734 


0.002 


0.539 


AF3 Aging Areas 


0.242 


0.209 


0.059 


AF1_SES 


0.041 


0.824 


0.002 


Institutional Characteristics (R 2 =.567, .409*) 








IF4_Local Revenue Support 


0.615 


0.010 


0.378 


IF2_College Size 


-0.400 


0.069 


0.160 


IFI College Costs 


0.161 


0.434 


0.026 


IF3 Educational Resources 


0.057 


0.781 


0.003 


Student Body (R 2 =.798, .747*) 








SF2 Full-Time White Students 


0.824 


0.000 


0.685 


SFI Needy Career Students 


-0.293 


0.044 


0.088 


SF3 Developmental Completers 


-0.166 


0.226 


0.030 


Stepwise Cross-Environment (R 2 =.7 55, .717*)** 








SF2 Full-Time White Students 


0.606 


0.005 


0.686 


AF2 Stable Families and Neighborhoods 


0.345 


0.077 


0.539 


NOTE: Except for the last equation, all equations shown 


are forced entiy except the last and are 


statistically significant at p<.05: shading indicates variables surviving in parallel stepwise equation. 
*Second R 2 is adjusted form **Equation parameters relaxed to allow step inclusion at p<.l level. 
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The forced entry service area factor regression explains around 60 percent of the GT 
variance (50 percent adjusted). Stable Families and Neighborhoods emerged as the prime 
predictor (Standardize Beta=.734), followed at a good distance by Aging Areas (.242). Area 
SES seemed to make no real impact on GT rate (.041), and neither of the last two factor beta 
tests for statistical significance were sufficient for stepwise equation inclusions. The betas of the 
forced entry regression of institutional trait factors ranked Local Revenue Support highest in 
predictive power (.615), but also found College Size to be a relatively important predictor (-.400, 
the smaller the college the higher the GT rate). The parallel stepwise regression, however, 
included only a single factor tenn — Local Revenue Support. Nevertheless, it was the student 
body factor set that formed the basis for the most robust equation (R 2 =.798, .747 adjusted). The 
factor with the highest beta (.824) was the one that combined full-time enrollment and low 
remediation need with white racial identity. The Needy Career Student factor also showed a 
statistically significant beta coefficient (-.293) of negative sign. Both of these showed up in the 
parallel stepwise equation, but the Developmental Completers factor failed to make the cut. 

The all-factor stepwise regression equation (with a slightly relaxed item inclusion 
criterion) had only two factor terms — Full-Time White Students (beta=.606, p=.005) and Stable 
Homes and Neighborhoods (beta=.345, p=.077). The equation R 2 puts their joint variance 
explanatory power at 76 percent (72 percent adjusted), indicating a formidable level of degree 
progress determination. The equation answers one of the questions about environmental levels 
posed earlier: Do service area demographics have any direct effects upon institutional 
performance, or do they impact student outcome rates mainly by determining student body 
constitution? The answer suggested by our regression analysis, adding some additional 
correlation data, is that both causal paths are significant. Stable Homes and Neighborhoods 
correlates strongly with Full-Time White Students (Pearson r=.643), showing it to be a prime 
conditioner of student body make-up, but its strong beta coefficient indicates that it also plays a 
modest but real direct role in community college performance. The equation furthermore speaks 
to lack of salience of institutional characteristics. School trait factors were conspicuous by their 
absence from the equation. The present analysis, however, is too primitive to undergird any 
confident assertion about level of causality. Working out the precise causal network of factors, 
including those measuring institutional effects, will take a full-scale structural equation path 
analysis, research beyond the scope of this exploratory paper. 

The final component of our research plan was an exploratory follow-up on Astin’s 
suggestion that the residuals of outcome explaining regression equations could be employed as 
superior assessment measures of institutional performance. We used the “best equation” just 
described to calculate the residuals for each Maryland community college (actual GT rate - 
expected GT rate given environmental factors). Then we tentatively interpreted these as 
measures of institutional effect relative to institutional environmental circumstance, and 
constructed Table 9 to assess the results. The left side of Table 9 displays actual Maryland 
community college GT rates in rank order (best to worst) and their expected GT rates according 
to the equation. The table’s right side shows the community colleges residuals and re-ra nk s 
institutions accordingly. Positive residuals imply doing better than expected performance and 
negative residuals tag underachieving institutions. 



44 




Table 9. Maryland Community Colleges Fall 2000 Cohort: 
Graduating or Transferring Rate — Actual versus Predicted 


Community College 


Rank 


Actual 


Predicted 


Community College 


Rank 


Residual 


Allegany C of MD 


1 


66.2 


61.9 


Howard CC 


1 


8.7 


Garrett CC 


2 


63.2 


61.6 


Wor-Wic CC 


2 


7.5 


Frederick CC 


3 


63.1 


56.3 


Frederick CC 


4 


6.8 


Howard CC 


4 


56.2 


47.5 


C of Southern MD 


5 


4.4 


Harford CC 


5 


54.5 


51.1 


Allegany C of MD 


3 


4.3 


Carroll CC 


6 


54.4 


59.2 


Harford CC 


6 


3.5 


Hagerstown CC 


7 


54.0 


55.2 


Garrett CC 


7 


1.6 


C of Southern MD 


8 


53.8 


49.4 


Montgomery C 


8 


1.1 


Cecil CC 


9 


51.6 


59.6 


Hagerstown CC 


9 


-1.2 


Montgomery C 


10 


50.6 


49.5 


Prince George's CC 


10 


-1.3 


Anne Arundel CC 


11 


47.8 


51.8 


Baltimore City CC 


14 


-3.2 


Wor-Wic CC 


12 


45.3 


37.8 


CC Baltimore Co 


11 


-3.9 


CC Baltimore Co 


13 


43.4 


47.3 


Anne Arundel CC 


12 


-4.1 


Chesapeake C 


14 


36.7 


47.9 


Carroll CC 


13 


-4.8 


Prince George's CC 


15 


29.1 


30.3 


Cecil CC 


15 


-8.0 


Baltimore City CC 


16 


23.0 


26.2 


Chesapeake C 


16 


-11.3 



Actual GT rank order and rank order based on GT relative to environmentally predicted 
GT are quite different. For example, the actual outcome ra nk for Wor-Wic Community College 
was 12, but its perfonnance relative to predicted outcome moved that institution into the number 
2 position among Maryland community colleges. Carroll Community College suffered the 
opposite fate, falling from rank 6 to rank 13. Prince George's Community College, on the other 
hand, moved out of second-to-last place up to tenth. Originally, top-ranked Allegany College of 
Maryland dropped to fifth place. Also, the overall linear association of actual GT and GT 
equation residual proved to be at best modest — Pearson r=.495 (r'=.245). 

Strong caveats are in order. Our “best equation” is a highly provisional construct derived 
from the analysis of too little data by some very problematic statistical techniques. The above 
exercise is only meant to suggest that it may be possible to use regression analysis to create a 
fairer and more realistic assessment methodology that takes into systematic account the 
environmental advantages and disadvantages of assessed institutions. 

A Unit Recode Data Illustration 

As a concluding note to the above aggregate level research, we would like to present 
some results from a unit record analysis on the interaction between student body characteristics 
and formal success outcomes. We were able to obtain parallel datasets containing Degree 
Progress and student attribute variables from two contrasting institutions — inner-suburban 
Prince George's Community College and rural Carroll Community College. To proceed, we 
selected from these files variables representing two of the most powerful factors identified by the 
cross-institution environmental impact analysis — credit load status (full-time/part-time) and 
college-readiness (remediation required or completed/not required), crossing them to produce 
four student status groupings shown in the row labels of Table 10. 
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Table 10. Carroll and Prince George's Community College Unit Record Data: 
Student Body Academic Status and Academic Success* 




Academic Status by 
Success 


Cohort by Academic 


Academic Status Group 


Status Group 


(Row %) 


Status (Column %) 


N 




CCC 


PGCC 


CCC 


PGCC 


CCC 


PGCC 


Developmental/Part-Time 


17.6% 


15.8% 


4.0% 


27.8% 


17 


366 


Developmental/Full-T ime 


27.8% 


25.9% 


18.4% 


43.7% 


79 


576 


College-Ready /Part-Time 


38.8% 


39.6% 


11.4% 


10.9% 


49 


144 


College-Ready/Full-Time 


66.7% 


51.5% 


66.3% 


17.5% 


285 


231 


Whole Cohort 


54.4% 


29.1% 


100.0% 


100.0% 


430 


1,317 



*Graduation and/or transfer 



The cross-tabulations of academic status by formal success (graduation or transfer within 
four years), displayed in Table 10’s leftmost data columns, reveal a quite similar pattern for each 
school. Both cohorts show the least academic achievement in the most at-risk 
Developmental/Part-time category (CCC 18%, PGCC 16%), then a steady increase of success 
down the groups, until we see a peaking for the most academically advantaged category — Full- 
time College-Ready students (67% and 52%, respectively). How then do we explain CCC’s 
much superior overall success rate (54% to PGCC’s 29%)? 

This turns out to be a straightforward reflection of the academic advantages of Carroll 
Community College’s student body, compared with that of Prince George's Community College, 
rather than different causal dynamics at the individual student level within the student bodies. 
The middle columns of Table 10 depict the clear differences in the student body constitution: 
CCC students in 2000 were miles ahead of Prince George's Community College student in 
academic preparation and study intensity. Almost two-thirds (66%) of the former were college- 
ready students studying full time, nearly four times the proportion PGCC students in this 
category (18%), while the latter’s largest academic status group turned out to be Developmental 
Non-Completer/Full-Time (44% compared with CCC’s 18%). Furthermore, PGCC had almost 
seven times the proportion of students in the most at-risk category — Part-Time, Developmental 
Non-Completers (28% to 4%). In fact, keeping the group rates constant, PGCC & CCC’s 
student body in terms of academic status distribution, simple mathematics would predict 44% 
success rate rather than 29% — a 25-point improvement and only 10 percent behind the 54% 
CCC rate. Obviously, these two schools are playing the institutional performance game on two 
drastically contrasting internal environment fields. 

Summary 

The new Degree Progress Model developed by Maryland’s community colleges provides 
an expanded set of student achievement indicators, including interim measures such as attaining 
sophomore status in good standing and continuing enrollment, which improves on the sole 
reliance on graduation rates still found in many accountability mandates. Examining these 
measures across institutions revealed considerable variability. Analysis of graduation-transfer 
rates for the 16 colleges found service-area and student body environmental effects. Finally, a 
best- fit regression was used to predict graduation- transfer rates for comparison with actual rates. 
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The research findings just summarized call into question the conventional focus on final 
outcomes indicators, especially sole reliance on graduation rate, as providing adequate grounds 
for institutional performance assessment. The data patterns demonstrate the advisability of 
considering a range of indicators, both final outcome (including transfer success), and pre- 
graduation indicators like sophomore status attainment and late persistence. Only a cross- 
indicator approach that tracks students through the academic process can enable college decision 
makers and oversight agencies to arrive at a realistic and useful assessment of degree progress at 
an institution. Assessment should be capable not only of judgment, but also more importantly, of 
yielding diagnostic insights pointing to needed institutional reforms. The data patterns also 
constitute an implicit criticism of environmentally contextless assessment. The discovery of a 
high level of impact upon cross-institutional academic outcomes by factors, like service area 
demographic, institutional characteristics, and student body traits, argues for the establishment 
and use of expected levels of institutional performance as a regular part of the assessment 
process. As a practical matter, cross-institutional analyses of multiple outcomes indicators and 
environment impacts a need for more accurate selection of institutional peer groups and in 
setting more realistic benchmarks for strategic planning indicators. 
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There is voluminous literature on retention of students in postsecondary institutions, 
especially targeting freshmen (Strauss & Volkwein, 2004; Gravely, 2003; Adehnan, 2006; Astin, 
1993; Pascarella & Terenzini, 1991). This research has tended to focus on the simple dichotomy 
of leaving versus staying at the college of initial enrollment. Only a few studies differentiate 
between students who left the institution but did not transfer, and those who left and transferred 
to another college. Unlike leavers who did not transfer, those who transfer have not rejected 
higher education; on the contrary, they have embraced it. These students have specifically 
rejected the institution of their initial enrollment, so much so that they are willing to undergo the 
inconvenience and cost of the university application procedure - and more importantly, the 
difficulty of acclimating to a new setting - a second time. 

Entering freshmen who indicate that they are contemplating transfer even before the 
academic term begins - those we tenn “at risk for transfer” - warrant particular attention. 
Considering the substantial amounts of time and money that parents and students use to select a 
college and those university officials allocate to recruiting and admission processes, one would 
expect the “right” fit between most new students and their chosen universities. More than one- 
fourth of students at four-year postsecondary institutions, however, do not complete their college 
education at the institution where they initially enrolled (Porter, 2002). Therefore, studying 
students who are at risk for transfer and identifying the pathways that distinguish between those 
who actually transfer and those who remain enrolled at the initial institution can advance our 
understanding of student persistence. It can further help university practitioners develop more 
effective strategies for managing recruitment, admissions, and the first few critical days and 
weeks after students’ matriculation. 

In explaining students’ retention, researchers have frequently relied on a “student- 
institution fit” perspective. The most well-known and widely used, Tinto’s model, describes the 
departure decision as depending on the degree of successful student integration into the academic 
and social structures of an institution (e.g., Cabrera, Castaneda, Nora & Hengstler, 1992; Kahn & 
Nauta, 2001; Hicks & Lerer, 2003). Academic and social integration are also thought to be 
influenced by factors such as family background, individual attributes, and pre-college 
preparation (Tinto 1975, 1987). Adding to the “student-institution fit” perspective are other 
studies suggesting that financial assistance and students’ certainty in their choice also play 
important roles in retention (Cabrera, Nora, & Castaneda, 1993; Porter, 2002). 
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